Content based user interface design

ABSTRACT

Design of a user interface can be based on content of a data source bound to the user interface. Access can be provided to a data source having fields containing respective items of data. The items can be automatically analyzed to determine which types of user interface elements and/or properties of user interface elements to bind to the fields. Information for binding the fields with the respective determined types of user interface elements and/or the properties can be generated or stored. A user interface design tool can have data field identifiers that identify data fields of a data source. Control-type designators associated with the data fields can allow a user to designate one of a plurality control-types. Property designators associated with one of the control-type designators can be included to allow a user to designate a property of the control-type designated by a corresponding control-type designator.

BACKGROUND

In computer programming, a common design paradigm or pattern is themodel-view-controller pattern. The model-view-controller pattern is usedto separate the design of a user interface from the substantive content,data, or model that it represents or manipulates. Integrated DevelopmentEnvironments (IDEs) such as Visual Studio, Sun Java Studio Creator, andothers allow programmers to design user interfaces somewhat independentof their data models.

One problem with implementing the model-view-controller pattern is thetime required for a programmer to design a view or user interface thatlogically matches the model or data source. Previously, IDEs and othersoftware tools for designing user interfaces have included some limitedfeatures for designing a view or user interface given a data source. Forexample, given a data source such as a database table, an IDE or designtool might use information describing the database table (e.g., aschema) to auto-create a number of user interface elements and associatethem with fields in the database table. For example, if the data sourcewere a table of books with fields such as author, title, etc., an IDE orlayout design tool might use schematic information to auto-createtextbox interface elements for the table's fields and label theinterface elements based on the names of the fields. Other rudimentarytechniques have been used to facilitate tailoring a user interface(control view, interactive form, web page, etc.) to a particular datasource. For example, allowing fields to be dragged onto a design surfaceand creating interface elements in response. However, even with thesetypes of tools, a programmer may still require significant time totailor a user interface for a particular data source.

Another problem with implementing the model-view-control pattern is thecomplexity of control logic and maintaining synchronicity between themodel or data source and the view with the view or user interface. Thisproblem has been reduced with the emergence of data binding techniques.Data binding allows a data source (e.g., an XML document, a databasetable, etc.) to be automatically bound with a user interface, eitherfrom data source to user interface, from user interface to data source,or both. This simplifies the programming model because the programmer,for example, does not have to write code to update each user interfaceelement with the data from the data source. Or, depending on the type ofbinding, the programmer may not have o write code to update the datasource when the user interface elements change. The behavior of theinterface and the content of the data source are linked by transparentmechanisms.

FIG. 1 shows a conceptual data binding example. A user interface or form50 is bound to data sources 52, in this case, an XML document 54(“books.xml”) and a database 56 with tables 58. The form 50 has userinterface elements such as scrolled lists 60 and textboxes 62, each ofwhich is bound to data in a data source 52. For example, a book's title(from XML document 54) might be bound to textbox 62, such that changesto one automatically cause changes to the other. An abstraction layer ofbinding sources 64 helps to hide details of accessing the data sources52. However, for purposes herein, a data source is considered to beeither an actual source of data (e.g., a database, an XML document, aservice provided by a server, etc.), or an object that handles orbrokers access to an actual data source, or a combination thereof. Databinding is available in a number of Java and .NET development tools.Documentation on how to program a data binding tool or component is alsoavailable.

However, it can be a slow and difficult process for a programmer totailor a user interface to a particular data source. For example, manyuser interface elements may need to be created, configured, and bound,element by element, to the data source. Furthermore, schema informationabout a data source may be needed, and changes in the configuration ofthe data source can require modification of the user interface.

SUMMARY

The following summary is included only to introduce some conceptsdiscussed in the Detailed Description below. This summary is notcomprehensive and is not intended to delineate the scope of protectablesubject matter, which is set forth by the claims presented at the end.

Design of a user interface can be based on content of a data sourcebound to the user interface. Access can be provided to a data sourcehaving fields containing respective items of data. The items can beautomatically analyzed to determine which types of user interfaceelements and/or properties of user interface elements to bind to thefields. Information for binding the fields with the respectivedetermined types of user interface elements and/or the properties can begenerated or stored. A user interface design tool can have data fieldidentifiers that identify data fields of a data source. Control-typedesignators associated with the data fields can allow a user todesignate one of a plurality of control-types. Property designatorsassociated with one of the control-type designators can be included toallow a user to designate a property of the control-type designated by acorresponding control-type designator.

Many of the attendant features will be more readily appreciated byreferring to the following detailed description considered in connectionwith the accompanying drawings.

DESCRIPTION OF THE DRAWINGS

Like reference numerals are used to designate like parts in theaccompanying Drawings.

FIG. 1 shows a conceptual data binding example.

FIG. 2 shows a process for configuring user interface elements.

FIG. 3 shows a detailed user interface design process.

FIG. 4 shows an example table of patterns for analyzing content or dataitems from a data source.

FIG. 5 shows how content analysis can be used to configure a userinterface design tool.

FIG. 6 shows how designators can be used to design a user interface.

FIG. 7 shows another form of user interface design tool.

FIG. 8 shows an example of information, in this case an XAML (ExtensibleApplication Markup Language) document, which can be generated forbinding a user interface to a data source.

DETAILED DESCRIPTION

FIG. 2 shows a process for configuring user interface elements. Theprocess can be performed by an IDE or user interface builder duringdesign time, or, if needed, the process can be performed at runtime by auser application that needs to bind to a data source as soon as the datasource is added. One way to improve the process of building a userinterface that is to be bound to a data source is to use the content ofthe data source to make intelligent choices about the user interface.Often times, the data type of the fields or elements of a data sourceare not known. For example, if the data source is an XML document and noschema is available, then the data types of the nodes may not be known.Or, the data source might be a flat file with character-delimitedcolumns with no available typing information. Again, even though thefields may actually carry a diverse range of data types, perhapshyperlinks in one column, numbers in another column, and currencies inanother column, by automatically inspecting the actual contents of thedata it becomes possible to make intelligent automatic user interfacedesign choices. It should be noted that content-based design choices canalso be useful even when typing information is available for a datasource. The data source could be a database and the data types of thecolumns may be known at design time. However, the data types provided bythe data source may not be accurate or precise. For example, a columnmay be a text data type and yet the column may specifically containhyperlinks, URLs, URIs, etc.

Referring again to FIG. 2, in one embodiment, user interface designprocess begins with acquiring 100 a data source. Once acquired 100, adata item or portion of content is read from a field of the data source.The data item is then analyzed 104 to determine a type of control oruser interface element. For example, based on analysis 104 of the dataitem, the process could determine that a textbox type of user interfaceelement is called for. There is no limit on the type of user interfaceelement that can be determined or selected; any current or future typeof control or user interface element can be chosen. Furthermore, thereneed not be a logical correspondence between the data item and thedetermined type of user interface. For example, although it might makesense to map a date-formatted data item a date-picker or some otherdate-specific user interface element, the data item could as easily bemapped to a textbox, or a slider, etc. Finally, the determined type ofuser interface element is actualized by generating 106 information forbinding or connecting the field that provided the data item with acontrol or interface element of the determined type.

FIG. 3 shows a detailed user interface design process. The user selectsor indicates 120 a data source, such as an XML document or stream, adatabase, or some other form of data source. The data source can beindicated 120 with a chooser-panel such as the Data Sources choosershown at the top of FIG. 5. Assuming that the data source isestablished, the process scans 122 the data source to identify fields ofthe data source. Information about fields (number of fields, fieldnames, a collection of fields, etc.) can be obtained, for example, froman object that wraps or presents the data source. The process theniterates through the fields by first getting 124 an unprocessed fieldand reading 126 content or a data item from the field. In oneembodiment, the data is then analyzed 104 by comparing 130 the data itemto successive patterns (see FIG. 4) until a match is found. Based on thematched pattern, a type of user interface element (e.g., textbox,button, slider, etc.) is determined 132. Preferably, a user interfaceelement type is determined 132 implicitly by using an interface elementtype that is associated with the pattern; the interface element type isassumed to be the desired type of user interface element that will beused for the data item's field.

The determined 132 user interface element type is then actualized bygenerating 133 information for binding or connecting the field with acontrol or interface element of the determined type. As seen in FIG. 3,this information can take many different forms. For example, thegenerated 133 information can be an XML or XAML (Extensible ApplicationMarkup Language) template 134 that expresses an object model using XML(see FIG. 8). In such a template 134, a user interface element isusually represented or declared by a corresponding type of XML element.The generated 133 information can also be in the form of a userinterface 136 for designing a view (user interface, interface layout,etc.), and more specifically, default settings of controls that can beused at design time to select different types of user interface elements(see the top interface in FIG. 6). The generated 133 information canalso be an actual instance 138 of a control or element of the determinedtype, bound to the gotten 124 field. In other words, the process of FIG.3 can be used at runtime to dynamically generate and bind user interfaceelements to the data source. These should not be considered the onlyways of generating 133 user interface information.

To iterate through all of the fields that will be processed, the processchecks to see if there are 142 more fields. If there are, the process isrepeated until there are no more fields to process, at which point theprocess is finished 144.

Regarding the processes in FIGS. 2 and 3, it should be noted that thegeneral idea of analyzing content for user interface design can be usedto chose user interface element properties as well as, or instead of,types of user interface elements. That is to say, other embodimentsanalyze data items from a data source to automatically select the userinterface element properties to bind to fields in the data source. Thistechnique is discussed in more detail later.

FIG. 4 shows an example table 160 of patterns for analyzing content ordata items from a data source. Table 160 has a column containingpatterns 162 that can be compared to data items. Table 160 also has acolumn containing user interface element types that correspond to thepatterns 162. It will be appreciated that the examples in table 160 arenot limited. Furthermore, the table 160 can be extended to include acolumn that identifies preferred user interface element propertiesassociated with respective patterns. It should also be noted that atable is only one of many ways that content can be analyzed. Anyheuristic or algorithm can be used to map data items to different typesor properties of user interface elements. Also, the patterns 162,whether in a table or in a coded algorithm, can be convenientlyimplemented using regular expressions. The table 160, or otherpattern-interface mapping, can be predefined, driven by context orinformation about the data source, or even modified by a user.Furthermore, more than one data item can be used to analyze a field. Forexample, many data items could be analyzed for a field, and the mostcommonly matching pattern can be used as the matching pattern for thatfield. Or, if an empty data item is found, other data items for thefield can be read until a non-empty item is found.

FIG. 5 shows how content analysis can be used to configure a userinterface design tool 180. A user may be presented with a data sourceselection panel 182, with which the user may indicate 120 a data sourcewhich can be acquired 100. Box 184 represents behind-the-scenes logic,such as the process in FIG. 3, for generating the design tool 180. Thedesign tool 180 is generated to have identifiers 186 (e.g. labels,checkboxes, etc.) that correspond to fields in the data source. If theidentifiers 186 are checkboxes, their selection controls whether thefield is to be included in the final user interface that is beingdesigned.

The design tool 180 is also provided with designators 188, which are anykind of user interface elements that a user can interact with to selector designate different types of user interface elements desired to bebound to the corresponding fields. The design tool 180 alternatively, oradditionally, is provided with property designators 190 for designatingproperties of user interface elements that are desired to be bound tofields. In this example, the designators 188 and 190 are dropdown lists,as shown in FIG. 6. The default designations of user interface elementtypes and/or properties of the designators 188, 190 are set inaccordance with content analysis in a manner similar to that of FIG. 3,and in particular steps 132, 133, and 136. For example, assuming thatthe “BookCoverImage” field's data item matched to the Image controlpattern (see FIG. 3), the designator for that field is set to a defaultuser interface element type of “Image”.

The user interface design tool 180 may also have a layout containerselector 192 for selecting a type of layout container that will be usedto contain the user interface elements upon completion of the interfacedesign. For example, a stack or grid layout container can be selected.The type of container affects how the user interface elements are laidout or arranged. As seen in FIG. 5, selection of a DockPanel containerresults in a layout as seen in preview area 194. The preview can changein accordance with changes to the selected container class.

FIG. 6 shows how designators 188 and/or 190 can be used to design a userinterface. As seen in the top part of FIG. 6, a designator 188 (dropdownlist 210) for the “Control Type” can be used to change the control oruser interface element type to something other than the default. Notethat in the upper example changing the control type for the“BookCoverImage” field to “TextBlock” results in a change in the previewarea 194: the user interface element 212 is now a TextBlock type controland is displayed accordingly. In the lower part of FIG. 6, a designator190 (dropdown list 214) is used to change a property to which thecorresponding field will be bound. If the type of user interface elementcontrol for a field is changed with a designator 180 or dropdown list210, then the properties for that field that are available for selectionwith designator 190 or dropdown list 213 will also change. For example,the items listed in dropdown list 214 will change.

FIG. 7 shows another form of user interface design tool 230. Fields orelements of a data source can be nested or hierarchical. Therefore, thisexample shows that the designators 188, 190 can also be nested. Forexample, if the “Children” field 232 has subfields such as “Name”,“Photo”, and “Age”, these subfields can also have correspondingdesignators 188 and corresponding user interface elements (per theirdesignators) in the preview area (not shown).

FIG. 8 shows an example of information, in this case an XAML (ExtensibleApplication Markup Language) document 250, which can be generated forbinding a user interface to a data source. The user interface designtool is configured to ultimately produce some information that can beused to display and run the designed user interface. As mentioned withreference to FIG. 3, this information can take many forms. However, withthe availability of declarative user interface technology such as XAML(Extensible Application Markup Language), based on XML, full blownsource code is not needed to define a user interface and its binding toa data source.

XAML allows separation of the presentation layer code from applicationlogic. XAML is a language for user interface programming. XAML allows adeveloper to control the layout of application interfaces includingtext, graphics, buttons, and user interface controls. Each XAML tagcorresponds directly to an interface class. A new button, for example,can be created by using a <Button> tag in an XAML file. The taginstantiates a Button object using the default constructor and allowssetting of properties of the button, such as text label, size, etc. XAMLcode can be used by a runtime language environment, a compiler, etc.

As can be seen in document 250, template information 252 can be used todefine user interface elements (e.g., a TextBlock element), and thoseelements can be bound to parts or fields of a data source (e.g., the“Title” field of the “BookData” data source). The template information252 can be in a separate file generated by the design tool. The outputof a design 180/230 can take other forms as well, such as actualinstantiation of a user interface, generation of source code, and so on.

In conclusion, those skilled in the art will realize that storagedevices utilized to store program instructions can be distributed acrossa network. For example a remote computer may store an example of theprocess described as software. A local or terminal computer may accessthe remote computer and download a part or all of the software to runthe program. Alternatively the local computer may download pieces of thesoftware as needed, or distributively process by executing some softwareinstructions at the local terminal and some at the remote computer (orcomputer network). Those skilled in the art will also realize that byutilizing conventional techniques known to those skilled in the art, allor a portion of the software instructions may be carried out by adedicated circuit, such as a DSP, programmable logic array, or the like.

All of the embodiments and features discussed above can be realized inthe form of information stored in volatile or non-volatile computer ordevice readable medium. This is deemed to include at least media such asCD-ROM, magnetic media, flash ROM, etc., storing machine executableinstructions, or source code, or any other information that can be usedto enable a computing device to perform the various embodiments. This isalso deemed to include at least volatile memory such as RAM storinginformation such as CPU instructions during execution of a programcarrying out an embodiment.

1. A method, performed by a processor of a computer, of selecting a typeand property of a user interface element to be associated with a datasource field, the method comprising: accessing a data source, the datasource comprising a plurality of fields, each field containing data;automatically analyzing, by the processor, the data in each field todetermine the type of the data; for each field, selecting a type of userinterface element and a property of the selected user interface elementto bind to the respective field based on the analysis of the data suchthat the type of user interface element and property of user interfaceelement that is selected for the field is based on a type of the data inthe field; displaying, in a user interface, a table that includes afirst column having a row for each field of the data source that wasanalyzed, a second column having a row for the user interface elementthat was selected to be bound to each field in the first column, and athird column having a row for the property of the user interface elementthat was selected to be bound to each field of the first column, whereinat least one of the rows of the first or second column displays anoption for changing the user interface element or property to which thefield is to be bound, and wherein the option is automatically set to acurrent selection based on the analysis of the data in the correspondingfield; displaying, in the user interface, a user interface control forselecting a layout container for containing each of the user interfaceelements that are selected to be bound to the fields being displayed inthe first column; and generating an XAML document that stores thebindings of the fields of the first column to the specified userinterface elements and properties in the corresponding second and thirdcolumns.
 2. A method according to claim 1, wherein the analyzingcomprises comparing the data items to predetermined patterns that areassociated with the types of user interface elements.
 3. A computingdevice configured to perform a method according to claim
 1. 4. Volatileor non-volatile computer-readable media storing information for acomputer to perform a process, the process comprising: designating adata source to be bound to a user interface for displaying andmanipulating data obtained from the source, the data source comprisingfields containing content; accessing the data source and automaticallyanalyzing, by the computer, the content of each field to automaticallyselect a type of user interface element and a user interface elementproperty to be bound to the field of the data source, wherein theselection of the type and property of the user interface element isbased on a type of the content in the field; displaying, in a userinterface, a table that includes a first column having a row for eachfield of the data source that was analyzed, a second column having a rowfor the user interface element that was selected to be bound to eachfield in the first column, and a third column having a row for theproperty of the user interface element that was selected to be bound toeach field of the first column, wherein at least one of the rows of thefirst or second column displays an option for changing the userinterface element or property to which the field is to be bound, andwherein the option is automatically set to a current selection based onthe analysis of the data in the corresponding field; displaying, in theuser interface, a user interface control for selecting a layoutcontainer for containing each of the user interface elements that areselected to be bound to the fields being displayed in the first column;and generating an XAML document that stores the bindings of the fieldsof the first column to the specified user interface elements andproperties in the corresponding second and third columns. 5.Computer-readable media according to claim 4, wherein the data sourcecomprises a database or a document comprising markup language andcontent identified thereby.
 6. Computer-readable media according toclaim 4, wherein generating the XAML document is responsive to userinput.
 7. A computing device comprising computer-readable mediaaccording to claim 4.